Add MoE prepare input kernels #29
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- restructure moe kernels folder - add prepare moe inputs kerel Signed-off-by: kareem <[email protected]>
kareemshaik80
changed the title
kareemshaik80
commented
Oct 30, 2025
Signed-off-by: kareem <[email protected]>
kareemshaik80
changed the title
Signed-off-by: kareem <[email protected]>
Signed-off-by: kareem <[email protected]>
Signed-off-by: kareem <[email protected]>
Signed-off-by: kareem <[email protected]>
Signed-off-by: Shaik, Kareem M <[email protected]>
kareemshaik80
changed the title
kareemshaik80
changed the title
Signed-off-by: kareem <[email protected]>
Signed-off-by: kareem <[email protected]>
airMeng
reviewed
Nov 11, 2025
airMeng
requested changes
Nov 11, 2025
msinnha1
reviewed
Dec 1, 2025
msinnha1
left a comment
msinnha1
left a comment
There was a problem hiding this comment.
One generic comment also to use error handling in the functions:
example, in void prepare_moe_input()
TORCH_CHECK(topk_ids.dtype() == torch::kInt32, "topk_ids must be int32");
...
Signed-off-by: Shaik, Kareem M <[email protected]>
Signed-off-by: Shaik, Kareem M <[email protected]>
kareemshaik80
changed the title
Signed-off-by: Shaik, Kareem M <[email protected]>
Signed-off-by: Shaik, Kareem M <[email protected]>
Collaborator
|
@kareemshaik80 please rebase with the latest main |
airMeng
reviewed
Dec 11, 2025
Collaborator
airMeng
left a comment
airMeng
left a comment
There was a problem hiding this comment.
generally follow SGLang CUDA kernel, LGTM except some minor comments
mingfeima
requested changes
Dec 11, 2025
Collaborator
mingfeima
left a comment
mingfeima
left a comment
There was a problem hiding this comment.
good job for this one!
just some minor places to change, and it shall be fine.
Collaborator
|
@kareemshaik80 could you please also collect how much is the ratio of |
Signed-off-by: Shaik, Kareem M <[email protected]>
airMeng
approved these changes
Dec 11, 2025
airMeng
reviewed
Dec 11, 2025
Collaborator
airMeng
left a comment
airMeng
left a comment
There was a problem hiding this comment.
Please add the new UT into https://github.com/sgl-project/sgl-kernel-xpu/blob/main/tests/run_suite.py
I forgot it too :)
Signed-off-by: Shaik, Kareem M <[email protected]>
mingfeima
approved these changes
Dec 12, 2025
Collaborator
@kareemshaik80 good work here! please continue to analysis how much overhead the shuffle takes, you can share data internally. |
airMeng
added a commit
that referenced
this pull request
Dec 12, 2025
kareemshaik80
added a commit
to kareemshaik80/sgl-kernel-xpu
that referenced
this pull request
Dec 12, 2025
…project#57)" This reverts commit eb9cfca.
airMeng
pushed a commit
that referenced
this pull request
Dec 12, 2025
* Revert "Revert "Add MoE prepare input kernels (#29)" (#57)" This reverts commit eb9cfca. Signed-off-by: Shaik, Kareem M <[email protected]>
sspintel
pushed a commit
to sspintel/sgl-kernel-xpu
that referenced
this pull request
Jan 2, 2026
* Restructure MoE and add prepare inputs/meta kernel - restructure moe kernels folder - add prepare moe inputs kerel Signed-off-by: kareem <[email protected]> * fix minor issues Signed-off-by: kareem <[email protected]> * Add tests Signed-off-by: kareem <[email protected]> * Add shuffle_rows Kernel Signed-off-by: kareem <[email protected]> * register shuffle_rows Signed-off-by: kareem <[email protected]> * Enable Build and Add apply_shuffle_mul_sum kernel Signed-off-by: kareem <[email protected]> * functional Signed-off-by: Shaik, Kareem M <[email protected]> * cleanup Signed-off-by: kareem <[email protected]> * cleanup1 Signed-off-by: kareem <[email protected]> * Modify fused expert to invoke moe_kernels and increase test coverage Signed-off-by: Shaik, Kareem M <[email protected]> * Cleanup makefile Signed-off-by: Shaik, Kareem M <[email protected]> * remove debug code Signed-off-by: Shaik, Kareem M <[email protected]> * fix lint Signed-off-by: Shaik, Kareem M <[email protected]> * Fix review comments Signed-off-by: Shaik, Kareem M <[email protected]> * Add to CI Signed-off-by: Shaik, Kareem M <[email protected]> --------- Signed-off-by: kareem <[email protected]> Signed-off-by: Shaik, Kareem M <[email protected]>
sspintel
pushed a commit
to sspintel/sgl-kernel-xpu
that referenced
this pull request
Jan 2, 2026
This reverts commit ac9e2a7.
sspintel
pushed a commit
to sspintel/sgl-kernel-xpu
that referenced
this pull request
Jan 2, 2026
* Revert "Revert "Add MoE prepare input kernels (sgl-project#29)" (sgl-project#57)" This reverts commit eb9cfca. Signed-off-by: Shaik, Kareem M <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.